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ABSTRACT 

A study and documentation of the first annual Award for 
Excellence in Teaching is made. A brief background for the 
Award is presented, including the formation of a committee 
designated to choose objectively a worthy recipient. The 
problems encountered by the committee in designing a success- 
ful ballot and the statistical methods used in selecting the 
recipient are of major concern and are treated in detail. 
Basic response statistics are published accompanied by a 
data summary. Special studies'^ including score analysis, 
weight component determination, and teacher elimination by 
dominance techniques are presented. To better understand 
the various population strata, several correlation studies 
are made . 
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I. 



BACKGROUND 



In May, 1969, the Superintendent of the Naval Postgrad- 
uate School, Radm. Robert W. McNitt, USN , established the 
Outstanding Teaching Award to be conferred upon a member of 
the Faculty of the Naval Postgraduate School annually at the 
time of the June Graduation Exercises. It was intended that 
the first conferral of the Award be made in June, 1970. 

Selection of the recipient was to be accomplished accord- 
ing to certain guideline procedures prepared by an Ad Hoc 
Committee appointed by the Academic Dean. The Ad Hoc Commit- 
tee consisted of Professors Gaskell (Chairman) , Handler, 
O'Toole, and Strum. In addition to basic guidelines, the Ad 
Hoc Committee made provisions for the establishment of a 
permanent selection committee to choose the recipient annually, 
thus insuring a growing efficiency and refinement of the pro- 
cess. Membership on the committee would be for two year pe- 
reiods , with exception of the first committee, and each year 
half of the committee would step down, thus combining past 
knowledge and familiarity of the problem with new talents 
and ideas . 

The first selection committee finally evolved to the 
membership of Professors Gawain (Chairman) , Kovach (Vice- 
Chairman and Secretary), Boggess , Nunn, Read, Strum, and 
Woehler. The initial efforts of the selection committee 
were towards setting a general tone for the award itself. 

It was felt that a small group of the Faculty members could 
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be identified who merit consideration for the Award, but 
that a selection from among this group of one individual, 
however deserving he may be, was not likely to preempt all 
other candidates in every respect. Therefore, it was felt 
that the selection of one individual could properly be re- 
garded as a representative and symbol of the excellence of 
his peers. 

In attempting to fulfill its assigned responsibilities, 
the committee tried to formulate an adequate philosophical 
outlook concerning the nature of excellence in teaching and 
how it could be reliably identified and evaluated. After 
much research, both outside and inside the committee, the 
Committee reached the following general conclusions : 

(a) There did not exist any objective and gener- 
ally accepted method for quantitatively measuring 
excellence in teaching. 

(b) Experience at the Naval Postgraduate School 
and elsewhere indicated that mature students and 
other qualified observers do on the whole show a 
reasonable degree of consistency in judgments of 
persons engaged in teaching. 

(c) The Committee could properly designate the 
recipient on the basis of the representative judg- 
ments of persons qualified to hold opinions. 

(d) It appeared that a polling procedure could 
be used to achieve this objective. 

(e) Well developed statistical techniques were 
available which could be used to guide the quanti- 
tative interpretation of the poll. 

The Committee next desired to put down on paper some 
statement that 'approached the correct interpretation of the 
phrase "excellence in teaching." This phrase was to .refer 
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to that complex of personal and professional qualities and 
actions on the part of the teacher which; 

(a) Make themselves felt primarily at the inter- 
face of personal contact between student and teacher. 

(b) Help transmute the student's encounters with 
his subject matter into insight, enlightenment and 
love of learning. 

(c) Elicit from the student responses in thought, 
feeling, and action which enhance his capacity for 

• self education. 

(d) Manifest themselves in an effective individual 
style which authentically reflects the teacher's 
own unique personality, experience, character, and 
convictions . 

With reference to the actual polling procedure to be 
followed, the Committee proposed the following specific points: 

(a) The following individuals would be eligible to 
participate; all students, all Faculty members,^ 
all Curricular and Assistant Curricular Officers, 
and a representative group of recent alumni. The 
exact composition of this group to be specified later. 

(b) Participation in the poll would be entirely 
voluntary . 

(c) The polling would be by secret ballot. 

(d) Stringent security measures would be taken to 
guarantee that the poll could not be misused in any 
fashion which would be detrimental to the status or 
interests of any individual Faculty member. 

(e) In order to be eligible to participate in the 
poll, an individual must have indicated that he con- 
siders himself to be reasonably well acquainted with 
the actual teaching performance of five or more 
Faculty menibers. 



^Department Chairmen and members of the Selection Committee 
were declared ineligible to receive the award. They were not 
excluded from the poll, however. 
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(f) Participants would be cautioned to avoid 
nominating teachers on the basis of mere personal- 
ity or popularity. No attempt would be made to 
provide a definite check list of required qualities 
for excellence in teaching. The Committee's guide- 
lines in this area were to be optional. 

(g) Data obtained in the poll would be analyzed 
objectively, using appropriate statistical methods 
which would take into account all relevant factors. 
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II. CONDUCT OF POLL 



A. BALLOT FORMAT 

After setting the philosophical tone of the award, the 
selection committee directed its attention to the appropriate 
ballot format. It was decided that the ballot itself would 
consist of three basic parts. The first, a statistical infor- 
mation section, was designed to obtain data to help understand 
various characteristics of the polled population. The second 
part consisted of a listing of the faculty eligible for the 
teaching award, and the third part was to invite a supporting 
remark for the voter's primary nomination. A sample ballot 
is included in Appendix A. 

Information requested in Part I of the ballot provided 
data for statistical procedures to be used in an effort to 
gain a better understanding of the voter population. It was 
desired to know the Voter Category of each individual being 
polled. Consequently, the four categories consisting of 
Student, Faculty, Alumnus, and Curricular Officer were used. 
These four categories were desired to be mutually exclusive 
and totally exhaustive. 

If the voter were military, additional information as to 
Rank and Branch of Service was requested. If the voter were 
a student, his curricular area would be of“ interest, and if 
he were a member of the Faculty, his Academic Rank and Depart- 
ment would supply pertinent statistical information. 
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The second part of the ballot displayed a listing of the 
eligible faculty, or those faculty who had engaged in teach- 
ing during the current academic year, except department chair 
men and members of the selection committee. 

An effective method of listing the eligible faculty was 
desired. The committee, realizing that most students (and 
consequently, alumnus) and faculty identified a teacher with 
an academic department, decided to have thirteen major heads 
corresponding to the thirteen departments. Within each de- 
partment, the teachers were listed alphabetically. Hopefully 
this method would facilitate finding a specified group of 
teachers with which the voter would be acquainted. 

Part III was included to obtain subjective information 
supporting the primary nomination of the voter concerned. 
Since this information was optional, the committee decided 
not to fix any policies regarding its usage in the selection 
process. But, such information would provide an inlet for 
subjectivity in the process which the committee felt might 
be necessary and appropriate. 

To accompany the actual ballot, a page of publicity and 
instruction was felt necessary. The committee decided that 
such a page should consist basically of two parts: a short 

statement of the background of the award ^d a statement of 
the balloting procedure. Therefore, a short summary of the 
philosophical thinking of the committee was included to 
introduce the subject of "teaching excellence" to the voter. 
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The balloting procedure consisted of five steps; three 
of which require explanation. Step 1 requested the voter to 
encircle the teacher number of all the faculty with which he 
was "sufficiently acquainted to make a judgment." It was 
emphasized that the voter encircle all those that fell into 
the above category in order to establish a basis of comparison. 

In Step 2, the voter was directed to select one (1) to 
three (3) nominees and to rank them in order of preference. 

The committee decided that a scoring system would be based 
on points awarded for first, second, and third place votes 
only . 

Finally, Step 3 invited the voter to utilize Part III of 
the actual ballot by giving supporting remarks for his first 
place nomination. This step was to be optional. 

B. VERIFICATION OF DATA 

The listing of teachers described above included a four 
digit number identifying each teacher. These numbers served 
a two-fold purpose. They were used in the processes of data 
collection and statistical manipulation; and secondly, the 
numbers served as a verification process by which the IBM 
data cards could be checked for key punch errors. 

Each teacher number was constructed in the following 
manner. The first three digits formed a nrimber that increased 
in numerical order for each teacher going down the listing, 
starting with number 101. Therefore, the second teacher down 
the list had the first three digits 102, the third teacher. 
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103, etc. Let N be a vector whose elements consist of 
these first three digits. The fourth digit of the teacher 
number was obtained by using the units digit of the inner 
product N • S' , where S was an arbitrary weight vector 
chosen to (351). As an example, for teacher 101, the 
fourth digit would be the units digit of 

(1 0 1) |s I or 4 . 

Thus, his full number would be 1014. This procedure rendered 
itself to computer verification of all key punched data cards. 
Other verification techniques appropriate to the programming 

I 

language (COBOL) were used for the statistical information 
provided in Part I of the ballot. 

C. RESPONSE ANOMALIES 

Although much effort was put into the design of the ballot 
format, it was anticipated that not every point would be clear 
to the individual voter. An analysis of the returned ballots 
proved this conjecture to be true. 

Multiple answers in any one of the informational areas of 
Part I would cause problems for proper statistical analysis. 

It was found that although the Voter Category information was 
totally exhaustive, it was not mutually exclusive, especially 
among the Alumnus and Curricular Officer categories. Conse- 
quently, revision of the collection process for this informa- 
tion is required. Confusion became apparent as to which 
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Military Rank category a foreign officer belonged.. Proper 
corresponding ranks should be outlined in any future polling 
of this sort. It was discovered that the options under 
Curricular Area were not mutually exclusive. An attempt was 
made to correct the above errors by examining the ballot con- 
cerned. Such errors were not used as a basis for ballot re- 
jection. 

The most common cause for rejection of a ballot was the 
requirement that the voter be acquainted with at least five 
faculty members listed as eligible teachers. This require- 
ment was established since the committee felt that each voter 
should be acquainted with at least a certain minimum number 
of teachers in order to make a meaningful comparison. The 
number five seemed reasonable. It would exclude most first 
quarter students but very few of the others. This rejection 
principal, although resulting in a "void" ballot, could not 
be considered an error on the part of the voter and conse- 
quently, was not a function of ballot format. The requirement 
was not publicized on the ballot, however. 

Two other major causes of rejection were late ballots and 
multiple selections for first, second, and third place votes. 
After a certain date, late ballots could no longer be consid- 
ered in the selection process and therefore became void. 

Also, there were many ballots returned blank because of the 
lack of forwarding addresses, A substantial number of voters 
interpreted the procedure instructions to mean that multiple 
choices for first, second, or third place nomination^ were 
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acceptable; thus, future clarification of this mis-interpre- 
tation is needed. Table I summarizes the void ballots. Late 
ballots consisted of alumni ballots received in the mail too 
late for processing. 

TABLE I 

Void Ballot Summary 
A. Late ballots 



Valid 


20 


Void 


2 


Blank (Returned 


37 


to sender) 




B. Pop. <5 


171 


C. Multiple Nominations 


24 



There were 1023 valid ballots used in the selection process. 
The results of the poll would become more representative and 
the statistical findings more meaningful with a reduction in 
the number of void ballots. 
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III. POPULATION CHARACTERISTICS 



A. RESPONSE STATISTICS 

One of the most interesting and important population 
characteristics concerned the degree of voluntary participa- 
tion the Teaching Award could arouse. In order to ascertain 
the level of interest reached by the voter population, basic 
response statistics were compiled. These statistics were 
based only on valid ballots; ballots that were free from the 
errors listed above and had ^ 5 , where was the size 

of the population of teachers ^identified by the i^h voter. 

Response by Voter Category gave general response statis- 
tics. Table II gives a two-way classification of the Student 
ballots received. Each interior entry in the table represents 
the number of ballots received from the i'^^ Military Rank 
of the j'th Curricular Area. The top row is a count of those 
Student ballots listing only a Curricular Area. Similarly, 
the left-hand column represents those ballots known only by 
Military Rank. The bottom row gives the percentage response 
within each Curricular Area. In review, from the 1485 stu- 
dents at the Postgraduate School when the poll was conducted, 
728 responded in an acceptable manner, yielding an overall 
response of 49%. 

Table III shows a similar response for* the Alumnus ballots. 
Each interior entry represents the number of ballots received 
from the i^^ Military Ranking of the Curricular Area. 
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TABLE II 



Response by Student Voter Category 



Student Curricular Area 



Military 

Rank 







Ops 

Ana 


Aer 

Eng 


Ele 

Eng 


Ord 

Eng 


Nav 

Eng 


Eng 

Sci 


Man 

CSc 


Env 

Sci 


Bac 

Pgm 


Def 

Man 






\p 

q\ 


4 


1 


9 


9 


4 


8 


3 


0 


2 


0 


R 


01 


0 


9 


6 


7 


5 


9 


5 


14 


2 


4 


0 


61 


02 


1 


0 


1 


3 


1 


3 


1 


1 


0 


0 


0 


11 


03 


0 


54 


25 


57 


25 


17 


44 


4 8' 


1 


30 


0 


301 


04 


2 


41 


10 


27 


23 


15 


39 


48 


0 


49 


1 


265 


05 


0 


6 


0 


1 


1 


1 


1 


22 


0 


21 


1 


54 


06 


1 


0 


0 


1 


0 


3 


0 


0 


0 


1 


0 


6 


S 




114 


43 


105 


64 


52 


98 


136 


3 


107 


2 




T 




256 


68 


153 


120 


79 


119 


319 


80 


291 


36 




U 




45 


63 


69 


53 


66 


82 


43 


4 


37 


6 





P Students known in Curricular Area only 
Q Students known in Military Rank only 
R Student Ballot Distribution by Rank 

S Total. Response by Curricular Area 
T Total Enrollment of Each Area 

U Percentage Response by Curricular Area 
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TABLE III 



Response by Aliimnus Voter Category 



Student Curricular Area 



Military 

Rank 







Ops 

Ana 


?^er 

Eng 


Ele 

Eng 


Ord 

Eng 


Nav 

Eng 


Eng 

Sci 


Man 

CSc 


Env 

Sci 


Bac 

Pgm 


Def 

Man 






\a 

B \ 


1 


1 


0 


0 


0 


1 


1 


0 


0 


0 


C 


01 


0 


1 


0 


1 


0 


0 ' 


0 


2 


0 


1 


0 


5 


02 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


03 


1 


1 


0 


4 


3 


1 


3 


6' 


1 


1 


0 


21 


04 


1 


14 


L2 


11 


19 


12 


21 


16 


1 


20 


1 


128 


05 


1 


8 


2 


2 


0 


0 


5 


12 


0 


L5 


1 


46 _ 


06 


0 


0 


0 


1 


0 


0 


0 


1 


0 


0 


0 


2 


D 




. 25 


15 


19 


22 


13 


30 


38 


2 


37 


2 





A Alumnus known by Curricular Area only 
B Alumnus known by Military Rank only 
C Alumnus ballot distribution by Rank 
D Total response by Curricular Area 
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The top row and left-hand column have the same meaning as 
above. The bottom row is the total response from the 
Curricular Area. Since there were two ballots that did not 
indicate a Military Rank or Curricular Area, the total Alumnus 
response was 208 ballots. There were 530 ballots mailed, 
yielding a return of 39%. 

It should be noted that the polling of Alumni poses some 
difficult problems. They must be contacted by mail, and hence 
a listing of current addresses is needed. This need can be 
served, to a large part, by the use of Reference 2 and the 
associated data tape which contains a' list of previous gradu- 
ates. There are shortcomings, however: 

(a) The tape contains listings of fomer officer 
students restricted to Naval personnel who are still 
active. Moreover, the tape requires periodic up- 
dating to keep the addresses reasonably current. 

(b) The addresses contained on the tape are coded. 
These must be translated into addresses that are 
usable in the mail system. Such can be accomplished 
with the aid of References 7 and 8, but experienced 
personnel are required to do this. 

Experience suggests that at least two months lead time are 
needed to update the tape, and at least one month to translate 
the addresses. 

Table IV shows the response of the Faculty ballots. Each 
interior entry represents the ratio 

# of ballots in the i,j^^ category . 
t^otal # possible 
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TABLE IV 



Response by Faculty Voter Category 



Academic Department 







Aer 

Ena 


Avi 

Sty 


Man 

CSc 


Ele 

Ena 


Gov 

Hum 


Ma 


Mec 

Ena 


Mr 


Oc 


Dps 

Ana 


Phy 


Mav 
Sy s 


Mat 
C he 




N^E 

f\ 


1 


1 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


Pro 


0 


2 

7 


0 

0 


2 

4 


3 

20 


0 

3 


3 

13 


0 

8 


0 

3 


2 

4 


0 

T 


8 

IT 


0 

4 


3 

10 


Acc 


0 


2 

7 


1 

2 


2 

8 


6 

19 


0 

3 


3 

6 


1 

2 


1 

4 


1 

2 


2 

12 


2 

13 


0 

3 


0 

3 


Ass 


0 


1 

3 


0 

1 


2 

6 


0 

T 


0 

T 


4 

5 


1 

6 


3 

3 


2 

6 


4 

16 


2 

5 


0 

1 


0 

1 


Ins 


0 


1 

2 


1 

2 


0 

8 


1 

3 


0 

2 


2 

12 


0 

1 


0 

3 


0 

2 


0 

4 


0 

3 


0 

8 


0 

0 


G 




7 

19 


3 

5 


6 

26 


10 

T6 


0 

12 


12 

36 


2 


4 

11 


5 

W 


6 


12 


0 


3 


H 




37 


60 


23 


22 


0 


33 


12 


31 


36 


17 


34 


0 


14 

21 



E 

F 

G 

H 



Faculty known in Academic Department only 
Faculty known in Academic Rank only 
Total response by Academic Department 
Percentage response of Academic Department 



thus, It IS the fractional response by Academic Rank from 
each Department. The top row and left-hand column again have 
the same interpretation as above. The bottom row gives the 
percentage response within each Academic Department. There 
were four (4) ballots received that did not list an Academic 
Rank or Department; thus, the total response from the Faculty 
was 74 ballots out of 289. The overall response of the 
Faculty was 25.6%, 

Of the ten Curricular Offices, a total of three ballots 
was received. 
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Of the 1023 valid ballots used by the selection committee 
the Voter Categories represented the following percentages: 

Curricular Office , 3 % 

Alumnus 20 3 % 

Faculty 72% 

Student * 72 ^ 2 ^^ 

Unknown 2 . i% 

B. DISTRIBUTION ANALYSIS OF N, AND M 

^ i 

In a further attempt to better understand the voter 
population, two characteristics were of interest. Associated 
with each ballot was the number of teachers identified by 
that ballot, and associated with each teacher was the number 
of voters identifying him. To gain this understanding, 
analyses on these two characteristics were made. 

Figure 1 is a histogram of N^ , the number of voters 
Identifying the kth teacher. The average value of Nj^ 
was 64.5, with a standard deviation of 49 . 4 . 

Figure 2 shows a frequency distribution of M. for 
the Faculty voters only; where M. is the number of teachers 
Identified by the ith voter. Similarly, Figure 3 is a 
distribution of for all the voters. 

It IS interesting to note the high average of the Faculty 
ballots, 32.1, when compared, to that of 15.5 for all the 
ballots. Undoubtedly the faculty’s criteria for "being ac- 
quainted" with a teacher are different from those of the 
students . 
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FIGURE 1 
Histogram of 
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FIGURE 2 



requency Distribution of 



For Faculty Ballots 




(All ballots) 

FIGURE 3 

Frequency Distribution of for All Ballots 
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C. DISTRIBUTION ANALYSIS OF 

Since an option was left to the voter whether or not to 
add a supporting statement, it was desired to know how the 
voter population reacted. The following analysis was made 
with this objective in mind. 

Define to be the number of ballots awarding a first 

place nomination to T^^ , and Dj^ to be the number of these 
that add a supporting statement. Figure 4 is a scatter dia- 
gram of Pj^ vs. ^ 5 , where Pj^ = , the 

proportion of that voluntarily added a comment. 

To estimate the probability that a supporting statement 
was made, a "pooled" value of resulted in 

P = I V = -341 

Examining Figure 4, it would be difficult to conclude 
that Pj, was in any sense approximately constant, or that 
any correlation coefficient other than zero would fit the 
observed data. It was concluded that the probability of 
receiving a supporting statement was independent of the 
number of first place votes. 
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IV. DATA SUMMARY 



The following is a summary of the data utilized by the 
committee in the selection process. Each row represents the 
pertinent data for a single teacher, and is ranked from top 
to bottom in order of decreasing score, where teacher k's 
score is defined to be 



• W , 



and W is the weighting vector analyzed below. and 

are as defined above, and , i = 1,2,3 are the number 

of first, second, and third place nominations received by 
teacher k . 
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V. SPECIAL ANALYSES 



The following special studies were conducted in order to 
objectively choose a worthy recipient and also to gain know- 
ledge concerning the distribution of positive scores. 

A. DOMINANCE CONCEPT AND SCREENING RATIONALE 

1 . Background 

In this section, a concept of dominance is developed 
and discussed which served as a powerful screening criterion 
leading to the final selection of the award winner. 

Let Tj^ represent teacher k- and Nj^ represent 
the number of voters indentifying teacher k on their ballots. 
Define as above. Now, let / i = 1/2,3 

represent the fraction of first, second, and third place votes 
awarded to T^^ from the population that knew him. Therefore, 
each teacher has a return vector, Vj^ = '^2k '^3k^ which 

will be used to measure his excellence as a teacher. Table V 
is a display of Vj^ for the top 25 teachers. 

2 . Development 

Teacher k's score, , can now be represented by 

3 

s, = y W.V., 

k 1 ik 



= + ”2^2k 






( 1 ) 
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TABLE V 



Return Vector 
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0.11538 
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0.08791 


0.07692 


6. 
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0.29630 
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0.23288 
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0.23595 


0.08989 
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0.07767 
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0.13725 


0.17647 


0.09804 


25 


0.19565 


0.07609 


0.04348 



where W = is a weighting vector to be specified 

later. 



In general, W is subject to the constraint that 



Wi > W2 > W3 > 0 . (2) 

It would be non-sensical to "give more weight to either a 
second or third place vote than to a first place vote. Con- 
sequently, from (2) above. 



36 



0 < ^ 
and 0 < w^/w 2 < 1 . 



( 3 ) 



A further restriction is necessary since values of 
and ^ 3/^2 near zero or unity are also undesirable. 

This would mean that either a second or third place vote re- 
ceived a weight of zero, or that they are weighted equally 
with a first or second place vote, respectively. Therefore, 
the inequalities in (3) above can be rewritten 



0 < a^ £ w^/w^ £ b^ < 1 

(4) 

and ^ ^2 — — ^2 ^ * 

Thus, (aj^,b^) and ^^2'^2^ define a preferential region 
interior to the unit square. Figure 5 shows .such a preferen- 
tial region when a^ = ‘a^ = 1/3 and “ ^2 “ * 




Preferential Region For 
a^ = a^ = 1/3 and b^ = b^ = 2/3 
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The shaded region represents points satisfying the general 
constraints in (3) above. The smaller square area DEFG 
depicts the preferred region where the ratios 
W 3 /W 2 satisfy the constraints in (4) . The region outside 
the larger square OABC violates the inequalities in (2) 
and (3) above. 

Concepts of dominance will now be developed. Let k 
and k' denote any two teachers. It is desired to find all 
possible values of W so that 






Therefore , 



''ihk + 



'"2^2k 






+ W-,V 



3''3k' 



or 






V^ki> + ”2^'^2k " ^^2k’^ ”a^^3k “ ^3'k’^ 



> 0 



(5) 



The first form of dominance becomes obvious from (5) 
above. When - ^ik' ^ ^ / i = 1/2,3 , then > Sj^, 

and , can be eliminated from competition. This form of 
dominance is termed absolute dominance. (See Table V) . 

Another form of dominance is readily generated 
following the observation that (5) defines a straight line in 
the two dimensional space of" Figure 5. F^om (5), 



Wi 



(V, 



Wz Ih 



- ^Ik') 



+ (V2k - V,^.) + 



2 k' 



W 2 ^ 3k 



“ V3k') 



> 0 



and 
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W3 > 
W2 — 



( 6 ) 



^ik - ^ik- ^ 
'^ak " ^3k' 



2k ^2k 
3k ^^3k 



where , ^ik' ' ^ ” 1/2,3 are known. The set (6) can 

be simplified to 



W3 

Wz 



> 



A 



kk' 



^kk’ 



(7) 



where A, , , and B , , are known constants. Consequently, 

KK KK 

the set (7) defines a half-space in which Sj^ dominates Sj^, . 




figure 6 

Case Examples' of Dominance^ 



Figure 6 is a plot of ^ 3/^2 * 

the old limits’ of ^2^^1 limits for 



Converting 

yields 
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( 8 ) 



1 < 3/2 £ ^ " 

These limits are plotted in Figure 6, together with four 
cases of dominance. 

In Case I, all the points that are feasible, those 
that satisfy (2) and (3) above, lie in the half-space where 
Sr > Sj^i . This is termed complete dominance. Absolute 
dominance is observed to be a special case of complete domi- 
nance. Case III may be called incomplete dominance; that is, 
where Sr > Sr, over portions of the feasible region, but 
not any of the preference region. In Case IV, the points 
necessary for Sr > Sr, are infeasible, resulting in Sr's 
elimination from competition. Case II represents the most 
interesting situation. In this case, both Sr and Sr, have 
the feasible and preferential ability to eliminate the other 
depending on the selected weight system. 

5. Results 

In the actual data, only thirteen teachers out of 249 
remained after an initial screening for absolute dominance. 

A secondary screeing for complete dominance reduced the group 
to four. A final screening for incomplete dominance further 
reduced the eligible teachers to a group of three. 

Figure 7 shows the results of this analysis. Let 
T^ , ^2 / and T^ be the teachers surviving the dominance 

screening criteria. From the data, S^ > S^ when 

> .73 ^ - 1.04 . 

W2 W2 
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FIGURE 7 

Results of Dominance Techniques 



This region is represented by the trapezoidal figure ABCD. 
Similarly, S 2 > when 

^ > .25 - 3.61 

W2 W2 



and ^ 1^23 ^hen 

^ > .68 — - 1.28 . 

W 2 ^W2 

These dominance lines generated areas where , *^2 ' 

T^ could win- the poll based on score alone, given that the 

point 'll fell into their respective regions. Several 

^ I W 2 W 2 . 
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further analyses of the data were made in an effort to choose 
among the three undominated teachers . 

B. PRINCIPAL COMPONENT ANALYSIS — DETERMINATION OF WEIGHTS 
1 . Background 

For purposes of preliminary analysis of the data, it 
was convenient to use tentative weights for first, second, 
and third place votes, and to rank the teachers according to 
the resulting scores. Several sets of weights were suggested 
by the Selection Committee (such as 3:2:1 and 4:2:1) and 
ultimately the system 4:2:1 was chosen for use in the dis- 
tribution and correlation studies. This selection was in- 
fluenced by the following analysis. 

It is reasonable to choose weights that would maximize 
the variance of the scores. This would facilitate ranking the 
teachers in a preferred order and reduce the number of scores 
very close together. In two dimensions, a teacher’s score 
could be represented by S = + w^x^ , which when graphed, 

becomes a straight line in the (x^,X 2 ) space. Figure 8 is 
such a plot. 

In three dimensions, these lines become planes and 
the objective in either case is to spread these lines or 
planes as far apart as possible. The method of principal 
components was used to obtain the set of weights that would 
accomplish this objective. 
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FIGURE 8 



Two-Dimensional Plot of (x^^x^). 



2 . Development 

Teacher k*s score, , can be represented by 



=k = = >■ — k 



W • V, 



where W is the desired weighting system. The objective is 
to choose W so as to Max ^Var (S^^) j . Since the scores 
could be made arbitrarily large by letting W be unbounded, 
a constraint on W is needed to solve the mathematical prob- 
lem. It is convenient to us^ the form 



W'W = 1 

to constrain the weight vector. 



(9) 
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Define V = 

value of the fraction of first 
taken over all teachers , 



^i 



Mo 



Mo 



k=l 



ik 



, where is the average 

second, and third place votes 



i = 1,2,3 



and where Mo is the total number of teachers identified by 
the ballots. The variance of the scores can be written 

Var(Sj^) = w' C W , (10) 

where C is the square, symmetric variance-covariance matrix 
of the , i.e. 

® ('^jk - Vk^"'"'' 

Combining the objective function and the constraint 
on W into the classical Lagrangian for the maximization 
problem yields 

L(W,X) = W' C W - X(W' W - 1) , (11) 

where X is the Lagrange multiplier. Equation (11) can be 
rewritten to facilitate obtaining the partial derivatives, 

3 3 3 

L(W,X) = I I w.c. .w. - X( ^ w? - 1) . (12) 

j=l i=l 1 



Taking the partial derivatives with respect to W 
and X and setting them equal to zero to obtain the maximizing 
value of W yields: 
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(13a) 



3L(W,X) 

8 w 



2CW-2XW=0 



and 



= W W - 1 = 0 



(13b) 



Rearranging (13a) , (C - XI) • W = 0 , results in a system of 
3X3 linear, homogeneous equations. Their solution is either 
trivial, W = 0 , or the coefficient matrix is singular. 



|C - Xl| = 0 . 



(14) 



It should be noted that (14) is the characteristic equation, 
a third degree polynomial in X , where the X's now also 
represent eigenvalues associated with the covariance matrix. 
From (13a) above. 



C W = X W , 
and Var (Sj^) = W C W 

= W' X W = X W' W , 

and using (13b) , Var(Sj^) = X . Thus, it follows that 
Var(Sj^) is maximized when X is maximized, and that the 
set of weights maximizing X is the eigenvector associated 
with X . Therefore, utilizing as the weighting system the 
eigenvector associated with the largest eigenvalue of the 
characteristic equation, results in a system that separates 
the scores as much as possible. 
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A fundamental fault with this approach concerns the 
satisfaction of the general and preferential restrictions 
on W . There is no guarantee that will occur 

or that the inner restrictions (aj^,bj^) and (a 2 /b 2 ) will 
be satisfactory. 

3 . Results 

From the data collected, 



C = 



.0087 


.0035 


.0017 1 


i 


.0035 


.0035 


.0015 




.0017 


.0015 


.0021 i 


1 


is X 


= .011 


, and the associated 


86, .45, 


.24) . 


This vector was modified 



to the simpler form W = (4,2,1) . Since both general and 

preferential limits were satisfied, (4,2,1) was used to 
weight first, second, and third place votes with the added 
knowledge that the resulting scores would be relatively well 
spread apart. 



C. PAIRED COMPARISONS ANALYSIS 

A special study was conducted on the top twenty teachers , 
where the top twenty was defined to be the twenty highest 
scoring teachers. This study consisted of comparing each 
pair of the twenty. For each pair, the score was based only 
on that sub-population of voters that identified both teachers. 

Define • • •' ^20 twenty teachers 

ranked in decreasing order of score. In this paired compari- 
son analysis, T^ would be expected to win over all others. 
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and T^q would be expected to lose. If lost in any of 

these tests, significant argument might be made for giving 
the award to another high ranking teacher, say T2 or . 

Again, let k and k' denote any two teachers and now 
let be the size of the population of voters knowing 

both teachers. Clearly, ” ^^kk'^ ” ^ * Table VI 

is a display of the square, symmetric matrix N . 

Define a new score, f to be the score of teacher k 

based on the sub-population, • Similarly, let 

be the score of teacher k' based on • Table VII 

is the square matrix of general, ^ ^kk' 

and the matrix in Table VII is not symmetric. 

Table VIII shows the results of such a paired comparison 
analysis for the top five teachers ranked as described above. 
Looking at the analysis for T^ , is defined as above, 

Tj^ is the teacher with whom is being compared, 

is the ratio of / where and are defined 

as above, and the plus sign is interpreted to mean that 

Si > Sk / and a minus to indicate a reversal, meaning . 

It is significant to note that T^^ loses twice and ties once 

under such an analysis. 

The importance of the size of was noted by the 

selection committee, and consequently, only reversals that 
occurred with larger values of were considered relevant. 

Table VIII is ranked under each teacher by order of decreasing 

«kk- • 
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TABLE VII 



S MATRIX 



1 


1 

0.0 


2 

0.0 


3 

1.26 


4 

2.00 


5 

0.0 


6 

2.00 


- 7 
2.77 


8 

0.0 


9 

0.0 


10 

0.0 


2 


0.0 


0.0 


1.23 


4.00 


0.57 


0.0 


1.14 


3.65 


2.15 


0.0 


3 


3.26 


0.46 


0.0 


1.33 


0.55 


2.00 


0.91 


0.0 


0.50 


2.00 


4 


1.00 


0.0 


0.0 


0.0 


1.00 


0.0 


0.83 


0.0 


0.0 


0.0 


5 


0.0 


0.57 


0.64 


2.00 


0.0 


2.00 


0.0 


0.0 


0.0 


2.00 


6 


2.00 


0.0 


1.33 


0.0 


0.50 


0.0 


0.0 


0.0 


0.0 


0.0 


7 


1.69 


0.14 


0.09 


0.67 


0.0 


0.0 


0.0 


0.0 


0.78 


1.00 


8 


0.0 


1.29 


0.75 


0.0 


0.0 


0.0 


0.0 


0.0 


0.67 


0.50 


9 


0.0 


0.38 


0.50 


0.0 


1.00 


0.0 - 


- 0.0 


0.0 


0.0 


0.0 


10 


0.0 


0.0 


0.0 


0.0 


3.00 


0.0 


0.0 


1.00 


0.0 


0.0 


11 


1.17 


1.33 


2.00 


1.50 


0.0 


0.0 


0.0 


0.33 


0.67 


0.67 


12 


0.75 


0.0 


0.45 


0.50 


1.50 


0.29 


0.13 


6.0 


0.0 


0.0 


13 


0.0 


0.0 


1.00 


2.00 


1.29 


0.0 


1.20 


4.00 


0.75 


0.56 


14 


0.41 


0.0 


0.73 


0.0 


0.0 


0.50 


0.0 


2 . 0 C 


3.33 


0.0 


15 


0.20 


0.0 


0.53 


0.0 


1.31 


0.0 


0.44 


2.00 


0.0 


0.0 


16 


0.0 


0.0 


4.00 


0.91 


0.0 


0.0 


1.00 


0.0 


2.00 


0.0 


17 


0.0 


0.0 


0.0 


0.50 


0.0 


0.0 


1.39 


0.0 


0.0 


0.0 


18 


0.0 


0.0 


0.45 


2.00 


0.40 


0.93 


0.29 


0.0 


0.0 


0.0 


19 


0.0 


0.0 


0.67 


0.0 


0.73 


0.0 


0.0 


0.0 


0.0 


0.0 


20 


0.0 


0.0 


0.0 


0.0 


1.50 


1.00 


0.0 


1.50 


0.0 


0.25 
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2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 



TABLE VII (CONT'D) 
S MATRIX 



u 


12 


11 




15 


11 


•JLZ 


11 


19 


20 


0.50 


3.00 


0.0 


0.94 


4.00 


2.00 


0.0 


2.67 


0.0 


0.0 


1.33 


1.33 


0.0 


0.0 


0.67 


0.0 


1.00 


1.00 


0.0 


0.0 


0.0 


2.29 


1.50 


3.47 


0.53 


0.0 


0.67 


1.85 


0.0 


0.0 


2.33 


0.0 


0.0 


4.00 


0.0 


2.00 


1.25 


0.0 


0.0 


0.0 


0.0 


0.17 


1.75 


4.00 


2.03 


4.00 


0.0 


0.0 


0.80 


1.00 


0.0 


2.29 


0.0 


1.70 


0.0 


0.0 


0.0 


1.87 


0.0 


0.0 


0.0 


0.13 


0.20 


0.0 


0.33 


1.00 


1.85 


0.14 


0.0 


0.0 


0.0 


0.0 


0.0 


0.50 


0.0 


0.0 


1.00 


0.0 


0.0 


1.03 


3.44 


0.80 


2.25 


0.33 


0.83 


1 . 00 - 


■ 0.0 


0.0 


0.0 


0.20 


1.33 


0.0 


2.00 


3.00 


2.00 


0.0 


0.0 


0.0 


0.0 


1.00 


0.0 


0.0 


0.67 


1.38 


0.0 


0.50 


2.00 


0.0 


0.0 


2.00 


0.0 


0.0 


0.25 


2.00 


2.30 


0.0 


0.33 


0.65 


2.91 


0.0 


0.0 


0.0 


0.0 


0.80 


0.88 


0.0 


0.0 


0.0 


0.0 


1.20 


0.13 


0.50 


0.0 


0.0 


2.00 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


1.15 


1.12 


0.0 


0.0 


0.0 


0.0 


0.29 


1.82 


0.0 


2.00 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.67 


0.0 


0.0 


0.0 


0.0 


0.0 


1.09 


0.67 


0.0 


0.29 


0.0 


0.67 


0.0 


0.0 


0.0 


0.0 


0.55 


0.0 


0.0 


1.20 


0.0 


0.0 


0.0 


0.0 


0.0 


0.13 


0.0 


1.60 


0.0 


0.0 


0.0 


0.0 


1.00 


0.0 


0.0 
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blank, signifies no contest 



D. SCATTER DIAGRAM ANALYSIS OF S, vs. N, 

k k 

An analysis was conducted on the relationship between 
Sj^ and , where Sj^ was the score received by teacher k 

and Nj^ was the voter population familiar with teacher k. 

It would be ridiculous to give the award to the teacher 
who received a perfect score because one person knew him and 
nominated him for first place. Thus, in this analysis, it 
was desired to identify a subgroup of high scores that were 
associated with small , if any existed; and if so, exclude 
these from competition. Figure 9 is a scatter diagram of 

vs. . From the diagram, it was concluded that this 

problem did not exist and therefore, a procedure for dissecting 
this space into two half-spaces, one that would be acceptable 
for further analysis and one that would be eliminated, was 
not researched. 

Another purpose of this analysis was to determine if any 
correlation of with Nj^ could be made, either positively 

or negatively. If it could be concluded that the regression 
coefficient did not equal zero, then it would have been neces- 
sary to find a different function of the polling data to serve 
as the score, one that was not correlated with Nj^ . 

Figure 9 again demonstrated that no correlation could be 
made and that the regression coefficient required would be 
zero, thus rendering the preceding analyses valid. 
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Nj^(# of ballots identifying Tj^ ) 
FIGURE 9 

Scatter Diagram of vs. Nj^ 
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E. SCORE DISTRIBUTION ANALYSIS 



Another special study was concerned with a determination 
of the distribution of the positive scores. With this know- 
ledge, probability statements and analyses could be made to 
a high degree of accuracy. The first step in this direction 
was the construction of a histogram of Sj^ (see Figure 10) . 
The histogram showed that the scores have a reverse-J shaped 
distribution. To estimate the form, the empirical tail- 
distribution was computed. The results are printed in Table 
IX, where 



and RTF = Tl/N is the relative tail frequency. 

From Table IX, a semi-logarithmic plot of score-cell mid- 
point against RTF was examined. Figure 11 was the resulting 
plot and it was concluded that a straight line could be mean- 
ingfully fitted to the data. Letting 



logio y = “ ,909x , and converting to the 

natural logarithm, 

.434 In y = - ,909x 



N is the number of positive scores 
fj is the frequency of the jth scoring-cell, 
TF is the tail frequency. 



and 



y = RTF 

X = score-cell midpoint 



or 



In y = -2.09x 



and therefore, y = e 



-2.09X 



Consequently 



P(X > x) = e 



-2'. 09x 



and this would yield an estimated 
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Sr (^=2:1) 



FIGURE 10 
Histogram of Sj^ 
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TABLE IX 



EMPIRICAL TAIL-DISTRIBUTION DATA 



Cell 




Cell Mp 


!i 


TF 


RTF(^) 


In RTF 


.001 


- 


.1 


.050 


42 


163 


.7955 


-.2295 


.101 


- 


.2 


.150 


34 


129 


.6295 


-.4630 


.201 


- 


.3 


.250 


23 


106 


.5170 


-.6600 


.301 


- 


.4 


.350 


27 


79 


.3855 


-.9550 


.401 


- 


.5 


.450 


13 


66 


. 3218 


-1.1350 


.501 


- 


.6 


.550 


13 


53 


.2585 


-1. 3500 


.601 


- 


.7 


.650 


5 


48 


.2342 


-1.4500 


.701 


- 


.8 


.750 


11 


37 


.1805 


-1.7130 


.801 


- 


.9 


.850 


7 


30 


.1463 


-1.9200 


.901 


- 


1.0 


.950 


7 


23 


.1122 


-2.1900 


1.001 


- 


1.1 


1.050 


1 


22 


.1073 


-2.2400 


1.101 


- 


1.2 


1.150 


3 


19 


.0927 


-2.3800 


1.201 


- 


1.3 


1.250 


5 


14 


.0683 


-2.6850 


1.301 


- 


1.4 


1.350 


1 


13 


.0634 


-2.7600 


1.901 


- 


1.5 


1.450 


1 


12 


.0586 


-2.8400 


1.501 


- 


1.6 


1.550 


1 


11 


. 0537 


-2.9200 


1.601 


- 


1.7 


1.650 


2 


9 


.0439 


-3.1300 


1.701 


- 


1.8 


1.750 


4 


5 


.0244 


-3.7200 


1.801 


- 


1.9 


1.850 


0 


5 


.0244 


-3.7200 


1.901 


- 


2.0 


1.950 


0 


5 


.0244 


-3.7200 


2.001 


- 


2.1 


2.050 


1 


4 


.0195 


-3.9300 


2.101 


- 


2.2 


2.150 


0 


4 


.0195 


-3.9300 


2.201 


- 


2.3 


2.250 


1 


3 


.0146 


-4.2400 


2.301 


_ 


2.4 


2.350 


3 


0 


.000 . 


00 



N = 205 
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log, „ (RTF) 
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density function of to be 

fg(s) = 2.09e"^-°^® , s >0 (15) 

The smooth curve on Figure 10 is a plot of this density 
function. 

It was desired to test the statistical validity of this 
density function as a true representation of the data. A 
Chi-Square Goodness-of-Fit Test was chosen. Table X is a 
collection of the data used in the Goodness-of-Fit test for 

Hq: fg(s) = 2.09e7^-°^® , s > 0 

Hq false, where 

1 ^, 1 ^, . . represent the scoring intervals, 

p^ = P(S e I^) under , Np^ is the expected number of 

entries in I. under H,, , and x. is the number of ob- 

served entries in I . . 

1 

k (x. - Np.)^ 

Define U = ) r; , where U is a ran- 

• T Np . 

1=1 ^1 

dom variable that approaches the Chi-Square Distribution func- 
tion with k-2 degrees of freedom as n->°° . Then, 
should be accepted with probability of Type I error equal to 
a , if the observed value of U , 

2 
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TABLE X 



Chi-Square Goodness-of-Fit Test Data 



i I. P. = P(S e I. ) Np. X. 

1 1 1 ^1 1 

under 



1 


{Sjs,^ < .1} 


.188 


38.6 


42 


2 


{Sj^l .1 < < .2) 


.153 


31.4 


34 


3 


{Sj,| .2 < £ .3} 


.124 


25.4 


23 


4 


{Sj^l .3 < Sj^ < .4) 


.101 


20.7 


27 


5 


• 


.081 


16.6 


13 


6 


• 


.067 


13.7 


13 


7 


• 


.054 


11.1 


5 


8 


• 


.043 


8.8 


11 


9 


• 


.035 


7.2 


7 


10 


{Sj^l .9 < Sj^ < 1.0} 


.030 


6.2 


7 


11 


{S^l 1.0 < Sj^ < 1.2} 


.042 


8.6 


4 


12 


{Sj^|l.2 < < 1.4} 


.028 


5.7 


6 


13 


{S^|1.4 < Sj^ < 1.7} 


.025 


5.1 


4 


14 


{Sj^lS^ >-1.7} 


.029 


5.9 


9 
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2 

From the observed data, u = 11,83 , and from the x 
distribution with twelve degrees of freedom and a = ,05 , 

X ^2 “ 21,0 , Therefore, 

2 

u < X2^2 ^0 accepted. 

Since the hypothesized density function passed the Chi-Square 
Test, it- was concluded that (15) above was an acceptable des- 
cription of the observed scores . 

F, CORRELATION ANALYSIS 

The correlation analyses below were designed to aid future 
committees by providing a better understanding of the voter 
population, 

1, Coefficient of Concordance Analyses ' 

Appendix B is a display of data used to estimate the 
effect of age of the voters. It was desired to know if the 
ratings of the teachers were somehow dependent upon the age 
of the individual voter. To accomplish this, Kendall's 
Coefficient of Concordance was applied to the ranking of 
sixty-two teachers by two separate subgroups of the voter 
population. Since military rank is, for the most part, pro- 
portional to age, it was felt that a ranking of the sixty- 
two teachers first by the 01 's , and secondly by the 04 's 
would give a wide enough spread in rank to encompass the 
effect of age upon voter nominations. Hence, Appendix B 
is a 2x62 matrix, where the first row represents the ranking 
of teachers by the 01 's , and the second row by the 04 's , 
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The choice of sixty-two teachers was made because of 
the following two restrictions. First, it was necessary to 
restrict the population of teachers to those known by both 
observer categories. Secondly, it was felt necessary to put 
a restriction on Nj^ , the number of ballots identifying 
teacher k , in each voter category. Consequently, each of 
the sixty-two teachers eligible for ranking were known by 
both the Ol's and the 04 's ; and in addition, at least 
five ballots from the population of Ol's and at least five 
from the 04's identified each teacher. 

From Kendall [Ref. 3] , 



W = 



m^ (n ^ 



12S 

nT) - 12m 'l t7 



0 < VI < 1 , 



where W is the Coefficient of Concordance modified to com- 
pensate for the existence of ties within rankings. The ob- 
served data yielded 



S = 69,504 
m =2 
n = 62 
T^ = 781 

^2 = 182 

and W = . 89 , 



The statistic W measures the communality of judgment be- 
tween the two rankings. Thus, a W close to one, such as 
the above, indicates a high degree of concordance between 
rankings, and* from the above test, it can be concluded that 
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age was not an important factor in how a voter rated the 
teachers . 

At this point, some measure of the significance of 
this observed value of W is needed. Such a measure is 
offered by Kendall, utilizing the Chi-Square distribution. 
Define 

u = 

mn(n + 1) 

= m(n - 1)W . Then U is approxi- 
mately distributed as (1 " “) with v = n - 1 degrees 
of freedom. From the observed data,’'u = 108.5 . Since 
V = 61 (> 30) degrees of freedom, 

/2)^^ ^ N(ll,l) 

or j - 11 N(0,1) 

or ( /2x^ ^ - 11 > z) = .01 , which implies 

- 11 > 2.23 . 

Thus, Xgj > 87.5 specifies the critical region. Therefore, 
the observed value of U , 

u = 108.5 > x^ = 87.5 , and it is concluded 
- 6 1 

that the resulting W above is substantially significant. 

For determining the effect of military rank on voter 
choice, a Kendall Coefficient of Concordance was computed for 
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the subpopulation of voters consisting of the OS's , 04 's , 

and OS's , a large group that had all ranked a common large 
number of teachers. The test was conducted under the same 
restrictions as above. Appendix C shows the 3x91 matrix of 
rankings. These data yielded 

W = .82 

and u = 222 . Consequently, 

u = 222 > = 122 , and therefore, from 

9 0 

this analysis, it was concluded that Military Rank did not 
influence the pattern of voter, nominations . 

A Coefficient of Concordance was developed for the 
rankings of the Faculty and those of the Students. It was 
desired to know if the Faculty in general, would vote as did 
the Students. Similar restrictions as those above were intro- 
duced to obtain more meaningful results. Appendix D displays 
the 2x193 matrix utilized in the test, with the following re- 
sults : 



W = .73 

and u = 280 . Consequently, 

u = 280 > = 238 , and W = .73 is 

19 2 

"just significant" at a = .01 . Therefore, it was concluded 
that both Students and Faculty generally agreed on their 
choices. 

Similarly, it was desired to know how the Alumnus would 
compare to the Faculty in such a test. Appendix E displays the 
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2x115 matrix of data. Restrictions such as those above were 
introduced with the following results; 

W = .65 

and u = 148 . Consequently, 

u = 148 < = 150 , and it was concluded 

114 

that W (= .65) was not significant at a = .01 . In other 
words , it, could have arisen by chance from a population where 
Faculty and Alumnus were completely unrelated. 

The tests of hypothesis for significance had as the 
null hypothesis and alternative, 

H^; W is not significant 
W is significant. 

At times, it would be more meaningful to know the value of a 
at which the decision regarding would just be changing 

from accept to reject, given the test statistic. This value 
of a is called the P-value [Ref. 1] and is commonly used 
as a substitute for the critical point. Thus, P = P (W ^ the 
observed W) . The P-value for the above calculations are 
determined as follows; 

Let u* be the observed value of the test statistic. 

Then 




where and are the mean and standard deviation of 

the Normal distribution approximation of the Chi-Square dis- 
tribution with V > 30 , and _ p 100(1 - P) 
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percentile of this Normal distribution. Now 



and P = 1 - _ p) , where N^^^l - 

is the value of the cumulative distribution function of the 
Standard Normal at _ p • 

The results of the Concordance Analyses with their 
respective P-values are summarized in Table XI below. 



TABLE XI 

Coefficient of Concordance Summary 



Observers 


W 


P-Val' 

-4 


01 *s and 04 's 


.89 


10 


OS's, 04's and OS's 


.82 


1 1 

o 
1 — 1 

V 


Faculty and Students 


.73 


< 10 


Alumnus and Faculty 


.65 


.018 



It was concluded that the Faculty and Students, in 
general, agreed upon their choices and that within the Students, 
agreement was high regardless of age or rank. The lower value 
for W , although not significant at a = .01 , noted in the 
test of Alumnus and Faculty would indicate that graduate views 
differ from current Students and Faculty. 

2 . Contingency Table Analyses 

It was desired to know if Academic Rank was associated 
with score. A contingency table was developed for Score Cate- 
gory vs. Academic Rank. This test could then conclude whether 
or not scoring was independent of the rank of the teachers 
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concerned; and consequently, whether or not the voter popu- 
lation favored one or more of the teacher categories. There- 
fore, it was desired to test 



H,. : P . . = w . s . 

0 ID ID 

against ^ '^i^j ' where 

Pij = P(Tj^ e (i,j)th cell) 

w^ = P e i^^ score category) 

and s . = P (T, e academic rank) 

D ^ 



TABLE -XII 

Contingency Table For 
Score Category vs. Academic Rank 



Academic Rank 



Score 

Category 





Prof 


Ac CO 


Asst 


Inst 


total 


.0001 


16 


14 


10 


8 


48 


.1200 


16.0 


17.3 


13.2 


6.6 




.1201 


11 


10 


11 


5 


37 


.2400 


11.1 


12.0 


9.2 


4.6 




.2401 


10 


21 


10 


4 


45 


.4300 


13.5 


14.7 


11.2 


5.6 




.4301 


12 


5 


9 


3 


29 


.7300 


8.8 


9.5 


7.2 


3.6 




.7301 


1.4 


13 


12 


6 


'*'45 


4.000 


13.5 


14.7 


11.2 


5.6 




total 


63 


68 


52 


26 


204 
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Table XII displays the data in a contingency table. 
Each interior cell has two entries. The upper entry repre- 
sents the observed number of teachers of academic rank j 
scoring in score category i . The lower entry of each cell 
is the number of entries that should be in the cell if 
is true. These numbers were found using the maximum likeli- 
hood estimators for w. and s. [Ref. 4]: 

1 3 



'7. = - y X. . 
1 n ii 



j=l 



- 1 ^ 

and S . = - y X . . 

1 n 13 



i = 1,2, 3, 4, 5 
j = 1>2,3,4 . 



X. . is the number of observed entries in each cell. Thus, 
ID 

from n sample values , 

nWj^Sj = expected number of entries 
in the (i,j)^^ cell. 

5 4 (X. . - nW.S.)^ 

Therefore, V = 7 7 3 

i=l j=l nW.S. 

is approximated by the Chi-Square distribution function with 
twelve degrees of freedom. From the observed data, 

V = 10.2 

and (- 95 ) = 21.0 . 

1 2 

Since v < (.95) , is accepted with 

12 U 



68 



probability of Type I error, a = .05 . It was concluded 
that the scoring distribution was independent of Academic Rank, 
and that the voter population did not prefer any academic rank 
in particular. 

A word should be said concerning the score categories 
used above. It was desired to have score categories such that 
each would encompass an equal number of expected entries; and 
in addition, that the expected number be greater than or equal 
to five. To achieve this, the exponential distribution esti- 
mated previously was used, through its inverse function, to 
arrive at five scoring intervals. 

TABLE XIII 

Contingency Table For 

Score Category vs. Student Curricular 'Area 





Ops 

Ana 


Ele 

Eng 


Ord 

Eng 


Nav 

Eng 


Eng 

Sci 


Man 

CSc 


Bac 

Pgm 


total 


. 0001 


9 


8 


7 


6 


12 


5 


6 


53 


.1200 


7.0 


8.7 


7.6 


5.7 


8.8 


8.8 


6.5 


.1201 


7 


9 


9 


6 


11 


9 


11 


62 


.2400 


8.2 


10.1 


8.9 


6.1 


10.3 


10.3 


7.7 


.2401 


9 


13 


10 


5 


14 


19 


11 


81 


.4300 


10.7 


13.2 


11.7 


8.6 


13.4 


13.4 


10.0 


.4301 


10 


17 


18 


11 


9 


10 


8 


83 


.7300 


11.0 


13.5 


1-2.0 


8.9 


13^7 


13.7 


10.2 


. 7301 


22 


23 


18 


18 


25 


28 


17 


151 


4.000 


20.0 


24.6 


21.8 


16.2 


25.0 


25.0 


18.5 


total 


57 


70 


62 


46 


71 


71 


53 


430 
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A similar analysis was made on Score Category vs. 
Student Curricular Area. Table XIII shows the resulting 
contingency table. Conclusions as to the independence of 
score category and individual student curricula were of 
interest. 

The test statistic now becomes 



V 



5 7 (X. . - nW.R.)^ 

y y -^3-- -- ^ -3 

i=l j=l nW.R. 

13 



where R. is the maximum likelihood estimator of r. , the 
D D 

probability of falling in the ' eurricular area. Thus 

V is approximated by the Chi-Square distribution function 
with twenty-four degrees of freedom. From the observed data 

V = 18.6 
(.95) = 36.4 

2 4 



Since v < (.95) , the null hypothesis of 

2 4 

independence is accepted with a = .05 . 

It was concluded that score category was independent 
of student curricular area, in that the probability of re- 
ceiving a score in one of the categories was not dependent 
upon which student curricula were sampled. 

Due to expeptionally low voter response in three cur- 
ricular areas, only seven curricula were represented in this 
analysis. It is to be noted that the contingency table 



70 



analyses are specific to the scoring intervals chosen. Time 
constraints precluded deeper analyses of these potential 
associations. 

It was desired to perform a Contingency Table Analysis 
on Score Category vs. Academic Department. Conclusions as to 
whether or not the scoring distribution was independent of 
academic department were of interest. Table XIV is a collec- 
tion of data for such an analysis. Due to the small expected 
and observed numbers involved, the analysis was not conducted 



71 



Table For Score Category vs. Academic Department 
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VI. CONCLUSION 



Members of the Selection Committee made individual eval- 
uation of the special studies presented above in preparation 
for their final choice of an award recipient. Following a 
presentation of the special studies, the committee cast a 
secret ballot to choose among the remaining three teachers. 

The names of individual teachers concerned were unknown 
to the Selection Committee throughout the process. At the time 
the above ballot results were revealed, the only name announced 
was that of the winner. His name was - forwarded to the Academic 
Dean and Superintendent, and will be made public at the June 
Graduation Ceremonies. 
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APPENDIX A 
SAMPLE BALLOT 



74 



AWARD FOR EXCELLENCE IN TEACHING 



Background - 



In order to augment the existing incentives for teaching of high quality, the 
Superintendent has established an award, called the Award for Excellence in Teaching. Having 
participated in the educational processes at the Naval Postgraduate School, your opinion is 
of value to the Selection Committee. In order to assess the collective opinion of students, 
faculty, staff, and selected alumni, the committee is conducting this poll. After accumulating 
the voting results the committee will select the Award recipient. 

The committee, mindful of the many measures of teaching excellence, does not wish 
to fix the rationale by which each voter determines his nominations. However, in order to 
assist those voters who desire something more definite as a guide to their thinking, the 
committee offers the following summary of some aspects of teaching excellence worthy of 
consideration: "The phrase, 'excellence in teaching,' refers to that complex of personal and 

professional qualities and actions on the part of the teacher which (a) make themselves felt 
primarily at the interface of personal contact between student and teacher; (b) help transmute 
the student's encounters with his subject matter into insight, enlightenment and love of 
learning; (c) elicit from the student responses in thought, feeling and action which enhance 
his capacity for self education and (d) manifest themselves in an effective individual style 
which authentically reflects the teacher's own unique personality, experience, character, 
and convictions." 

A meaningful selection of the Award recipient is heavily dependent upon the receipt 
of nominations from a large portion of the eligible "voters." Your response is therefore 
earnestly requested. Please complete the enclosed ballot in accordance with the attached 
instructions. The results of the polling, other than the Award recipient, will be treated 
as privileged information by the committee. 

BALLOTING PROCEDURE: 

Step 0. Complete the information requested at the top of the ballot. These items are for 
purposes of statistical analyses only. 

Step 1. On the list of eligible faculty encircle the number at the left of those with whom 
you are sufficiently acquainted to make a judgment. Use your own guidelines. For 
purposes of establishing a population base, it is important to encircle all with 
whom you are acquainted. 

Step 2. From the subset of faculty you have indicated in Step 1, select from one (1) to 
three (3) nominees. Indicate the order of your preference by placing a number 1, 

2, or 3 opposite and to the left of the appropriate name. Numeral "1" indicates the 
first choice, etc. 

Step 3. You are invited to furnish a short statement in support of your primary nomination. 
Space is provided for this at the and of the ballot. 

Step 4. Return the ballot by April 7. 

Students: Return to your curricular officer. 

Faculty and Staff: Send to Code 55 Bal. 

Alumni: Return via the enclosed return envelope. 
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BALLOT 



Part I - Statistical (encircle code at left) 

S Student 
F Faculty 

Voter Category A Alumnus 

C Curricular Officer 



Military Civilian 



Rank 


Branch of Service 


Curricular Area 




Rank 




Oepartment 


01 


N 


Navy 


30 


Ops Analysis 


IR 


Instructor 


AE 


Aeronautics 


02 


M 


Marine Corps 


31 


Aero Eng 


AT 


Assist Prof 


AO 


Aviation Safety 


03 


A 


Army 


32 


Elec & Comm Eng 


AC 


Assoc Prof 


Mil 


Bus Ad S Econ 


04 


C 


Coast Guard 


33 


Ordnance 


PR 


Professor 


EE 


Elec Eng 


05 


L 


Air Force 


34 


Naval Eng 






GH 


Govt & Humanities 


06 


F 


Foreign 


35 


Env Sci 






MC 


Mat Sc i Chem 


07 






36 


Manage & Comptr Sci 






MA 


Math 








37 


Eng Sci 






ME 


Mech Eng 








38 


Baccalaureate 






MR 


Meteorology 








39 


Oef Management 






OC 


Oceanography 
















OA 


Ops Analysis 
















PH 


Physics 
















NS 


Navy Manage Sys Center 



Part II - List of Eligible Faculty (all faculty engaged in teaching in the academic year 1969-70 except department 
chairmen and members of the selection conmittee) 



Aviation Safety Programs 


1319 


Lande, R. S. 


1658 


Parker, S. R. 


1014 


Bomberger, R. B. 


1320 


Lane, M. L. 


1669 


Sackman, G. L. 


1025 


Bradbury, C. M. 


1331 


Lee, M. 


1670 


She ingold, A. 


1036 


Fletcher, J. L, 


1342 


Musgrave, G. L. 


— 1681 


Smith, W. C. 


1047 


Wible, L. C. 


1353 


Senger, J. 0. 


1692 


Stentz, 0. A. 






1364 


Steckler, M. J. 


1708 


Terman, F. W. 


Department of Aeronautics 


1375 


Wegener, W. h. 


1719 


Thaler, G. J. 


1058 


Ball, R. E. 


1386 


Womer, N. K. 


1720 


Turner, J. B. , Jr. 


1069 

1070 


Bennett, J. A. J. 
Biblarz, 0. 


Oepartment of Electrical Engineering 


1731 

1742 


Vivell, A. E. 
Ward, J. R. 


__ 1081 

1092 

1108 


Collins, 0. J. 
Oelph, T. J. 
Haupt, U. 


1397 

1403 

1414 


Badger, R. R. 
Bauer, W. M. 
Baycura, 0. M. 


1753 Wilcox, M. L. 

Oepartment of Government 
and Humanities 


1119 

1120 

1131 

1142 

1153 

1164 

1175 

1186 


Kahr, C. H. 
Layton, D. M. 
Lindsey, G. H. 
Miller, J. A. 
Netzer, 0. W. 
Schmidt, L. V. 
Vavra, M. H. 
Zucker, R. 0. 


1425 

1436 

1447’ 

1458 

1469 

1470 

_ 1481 

1492 

1508 


Borst, F. W. , Jr. 
Bouldry, J. M. 
Breida, S. 
Campbell , J. 0. 
Chan, S. G. 
Chaney, J. G. 
Cooper, P. E. 
Cotton, M. L. 

Oe Laura, R. 0. 


1764 

1775 

1786 

1797 

1803 

1814 

1825 

1836 


Alexander, W. P. , Jr. 
BJarnason, L. L. 
Gabel, B. B. 
Gottschalk, S. 

Huff, B. F. 

McAdams, J. M. 
Pearson, L. W. 

Stolfi , R. H. 


Department of Business Administration 
and Economics 


1519 


Oemetry, J. S. 


1847 


Teti, F. M. 






1520 


Ewing, G. 0. 


Oepartment of Material Science and 


1197 


Carrick, P. M. 




Gardner, E. M. 




Chemistry 












1203 


Castro, B. 


1542 


Gerba, A. , Jr. 


1858 


Clark, J. R. 


1214 


Church, W. H. 


1553 


Hoisington, D. V. 


1869 


Helliwell , R. W. 


1225 


Cowie, J. B. 


1564 


Houston, R. K. 


1870 


Hering, C. A. 


1236 


Oarbyshire, L- 


1575 


Kirk, 0. E. 


1881 


Kinney, G. F. 


1247 


Eisenhardt, P. 


1586 


Klamrn, C. F. , Jr. 


1892 


Marshall , G. D. 


1258 


Elster, R. S. 


1597 


Marmont, G. H. 


1908 


McFarlin, G. H. 


1269 


Fremgen, J. M. 


1603 


Miller, R. L. 


1919 


Reynolds, M. F. 


1270 


Ganz, J. G. 


1614 


Murray, R. P. 


1920 


Rowell , C. F. 


1281 


Githens, W. H. 


1625 


Myers, G. A. 


1931 


Schultz, J. W. 


1292 


Hoverland, H. A. 


1636 


Myers, H. L. 


1942 


Sinclair, J. E. 


1308 


Jolly, J. A. 


1647 


Oler, C. B. 


1953 


Tolies, W. M. 










1964 


Wilson, J. W. 
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Department of Mathematics 
_ 1975 Barksdale, G. L. 

1986 Bender. A. P. 

1997 Bleick. W. E. 

2006 Bolles, R. C. 

2017 Sralnerd, W. S. 

202B Calabrese, P. G. 

__ 2039 Oavis. 0. L. 

2040 Oeaton, L. w. 

2051 Oixon, 0. R. 

2062 Estell.R. J. 

2073 Faulkner, F. 0. 

2084 Giarratana, J. 

2095 Hunt. R. W. 

2101 Jayachandran , T. 

2112 Jennings , W. 

2123 Kildall , G. A. 

2134 Kodres, U. R. 

2145 Kolitz, B. L. 

2156 Little. W. A. 

2167 Litzler, L. G. 

217B Lucas, K. R. 

2189 Marks. H. B. 

2190 Morris, G. W. 

2206 Pierce. J. P. 

2217 Preisendorfer, R. w. 

2228 Pulliam, F. M. 

2239 Roberts , A. B. 

2240 Schwarzkopf, A. B. 

2251 Shorb, A. M. 

2262 Singer, E. A. 

2273 Spalding. J. H. 

2284 Stewart, E. J. 

2295 Trahan, 0. H. 

2301 Weir. M. D. 

2312 Wilde, C. 0. 

Oepartnent of Mechanical Engineering 

2323 Brock, J. E. 

__ 2334 Cantin, G. 

2345 Houlihan, T. T. 

2356 Kelleher, M. 0. 

2367 Lynch. E. F. 

2378 Marto, P. J. 

2389 Newton, R. E. 

2390 Nguyen, 0. H. 

2406 Prowell , R. W. 

2417 Pucci , P. F. 

242B Tyvand, N. P. 

2439 Windrey, R. C. 

Department of Meteorology 

2440 Alberty, R. L. 

3476 Carrigan, R. C. 

2451 Outhie, W. D. 

2462 Elsberry, R. L. 



_ 3487 Hamilton, H. 0. 

_ 2473 Mahlman, J. 0. 

_ 2484 Martin, F. L. 

_ 3498 Oakes, U. 

2495 Renard , R. J. 

_ 2501 Taylor, C. L. 

2512 van der Bi j1 , W. 

_ 2523 Williams, R. T. 

Department of Oceanography 
_ 2534 Andrews, R. S. 

2545 Boston, N. E. J. 

_ 2556 Crew. H. 

_ 2567 Oenner, W. W. 

_ 2578 Geary, J, E. 

_ 2589 Giles, C. F. 

_ 2590 Haderlie, E. C. 

_ 2606 Jung, G. H. 

_ 2617 Smith, R. J. 

_ 2628 Thompson, W. C. 

_ 2639 Thornton, E. B. 

_ 2640 Tucker, S. P. 

_ 2651 van Schwind, J. J. 

_ 2662 Wickham, J. B. 
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APPENDIX B 

COEFFICIENT OF CONCORDANCE DATA FOR 01 's AND 04’ s. 
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19 
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52 
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52 


30.5 
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10 


25 


27 


13 


19 
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30 
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11 


12 


13 


14 


15 


16 


17 
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14 
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20.5 
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34 


36 


46 
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39 
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APPENDIX D 

COEFFICIENT OF CONCORDANCE DATA FOR FACULTY AND STUDENTS 
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91 


92 
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97 


98 
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77 
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21 
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76 


90 


82 
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36.5 
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2.5 
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Students 
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43 
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9 


110.5 


146 


5 


13 


156 
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122 


123 


124 


125 
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Faculty 


136 


39 


136 


22 


136 


136 


18.5 


136 


136 


Students 


176.5 


22 


154.5 


44 


81 


157 


23.5 


61.5 


113 
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129 
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133 


134 


135 


Faculty 
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9 
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41 


136 


136 
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176.5 


125 


1 


119 


70 


10 


97 


176.5 
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79 
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20 
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47.5 


54 
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93 
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50 
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86 
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176.5 
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151 
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136 
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45.5 


24.5 


29.5 
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Students 


176.5 


149 


36 


122 


176.5 


7 


11 


137 


125 
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157 


158 


159 


160 


161 


162 


Faculty 


24.5 


59 


136 


136 


136 


7 


62 


50.5 


136 


Students 


103.5 


32 


14 


160 


89 


64.5 


67 


110.5 


54 
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164 
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166 


167 


168 


169 


170 


171 


Faculty 


136 


43.5 


49 


43.5 


136 


4 


136 


136 


82.5 


Students 


94 


35 


151 


74 


95 


3 


176.5 


69 


17 




172 


173 


174 


175 


176 


177 


178 


179 


180 


Faculty 


74 


136 


136 


72.5 


136 


82.5 


136 


27 


59 


Students 


128 


101 


98 


145 


72 


78 


132 


38 


64.5 




181 


182 


183 


184 


185 


186 


187 


188 


189 


Faculty 


14.5 


136 


136 


75 


136 


71 


42 


136 


136 


Students 


96 


123 


85 


80 


176.5 


176.5 


63 


176.5 


176.5 
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191 


192 


193 












Faculty 


136 


136 


136 


136 












Students 


176.5 


148 


176.5 


176.5 
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APPENDIX E 



COEFFICIENT OF CONCORDANCE DATA FOR FACULTY AND ALUMNI 
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2 


3 


4 


5 


6 




7 


8 


9 


Faculty 


1 


59 


88.5 


54 


26 


59 




21.5 


19.5 


4 


Alumnus 


10 


83 


21 


18.5 


18.5 


41. 


5 


83 


43.5 


41.5 




10 


11 


12 


13 


14 


15 




16 


17 


18 


Faculty 


51 


46 


3 


18 


78.5 


33 




69 


105 


62 


Alumnus 


83 


83 


1 


43.5 


83 


39 




83 


83 


83 




19 


20 


21 


22 


23 


24 




25 


26 


27 


Faculty 


105 


32 


105 


34.5 


105 


30 




43 


73 


16.5 


Alumnus 


83 


83 


83 


35.5 


83 


83 




37 


32.5 


83 




28 


29 


30 


31 


32 


33 




34 


35 


36 


Faculty 


19.5 


46 


52 


105 


29 


41. 


,5 


59 


82 


105 


Alumnus 


9 


49 


83 


83 


23 


*52 




83 


83 


53 




37 


38 


39 


40 


41 


42 




43 


44 


45 


Faculty 


7 


93 


31 


90.5 


105 


40 




53 


49 


11 


Alumnus 


5 


83 


83 


83 


83 


17 




83 


83 


29.5 




46 


47 


48 


49 


50 


51 




52 


53 


54 


Faculty 


46 


105 


59 


59 


105 


39 




6 


25 


27 


Alumnus 


83 


83 


83 


83 


83 


6 




83 


83 


83 




55 


56 


57 


58 


59 


60 




61 


62 


63 


Faculty 


41.5 


105 


105 


85 


12 


38 




76 


75 


90.5 


Alumnus 


7.5 


83 


83 


83 


45 


15 




83 


83 


34 




64 


65 


66 


67 


68 


69 




70 


71 


72 


Faculty 


105 


80 


85 


46 


10 


105 




23 


46 


82 


Alumnus 


50 


83 


83 


83 


83 


12. 


.5 


21 


83 


83 




73 


74 


75 


76 


77 


78 




79 


80 


81 


Faculty 


14 


56 


78.5 


5 


34.5 


77 




74 


8 


50 


Alumnus 


21 


83 


83 


11 


83 


83 




83 


4 


83 




82 


83 


84 


85 


86 


87 




88 


89 


90 


Faculty 


13 


105 


87 


28 , 


105 


92 




15 


82 


36.5 


Alumnus 


24 


51 


. 83 


29.5 


35.5 


83 




28 


12.5 


16 



83 





91 


92 


93 


94 


95 


96 


97 


98 


99 


Faculty 


66 


36.5 


16.5 


9 


64.5 


105 


105 


105 


85 


Alumnus 


83 


39 


83 


3 


83 


32.5 


83 


26.5 


31 




100 


101 


102 


103 


104 


105 


106 


107 


108 


Faculty 


24 


105 


2 


71.5 


71.5 


63 


67 


68 


94 


Alumnus 


26.5 


83 


2 


54.5 


48 


83 


47 


54.5 


83 




109 


110 


111 


112 


113 


114 


115 






Faculty 


88.5 


105 


70 


64.5 


21.5 


105 


55 






Alumnus 


14 


39 


7.5 


83 


83 


46 


25 
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